[scout] move internal api tests to scout steps#263021
[scout] move internal api tests to scout steps#263021dmlemeshko merged 6 commits intoelastic:mainfrom
Conversation
| // Clean up output | ||
| if (outputId) { | ||
| await apiServices.fleet.outputs.delete(outputId); | ||
| apiTest.describe('Fleet Outputs Management', { tag: [...tags.stateful.classic] }, () => { |
There was a problem hiding this comment.
Why was the tags.serverless.security.complete removed?
There was a problem hiding this comment.
API was returning 400, we don't have SecSol tests for now. I think tag was added by default and we never actually tested it
⏳ Build in-progress, with failures
Failed CI StepsHistory
|
|
Starting backport for target branches: 8.19, 9.2, 9.3, 9.4 https://github.com/elastic/kibana/actions/runs/24427374909 |
## Summary Closes elastic/appex-qa-team#850 With selective testing enabled, it makes sense to move scout internal api tests running together with other tests, so that they can be skipped completely when there are 0 affected modules. I also noticed that running tests without building plugins might be not reliable, so let's run internal tests against build as any other tests. <img width="1132" height="467" alt="Screenshot 2026-04-14 at 15 35 28" src="https://github.com/user-attachments/assets/1cc6453a-74ad-45be-b855-ecccc28fc20f" /> (cherry picked from commit a34ca6b)
## Summary Closes elastic/appex-qa-team#850 With selective testing enabled, it makes sense to move scout internal api tests running together with other tests, so that they can be skipped completely when there are 0 affected modules. I also noticed that running tests without building plugins might be not reliable, so let's run internal tests against build as any other tests. <img width="1132" height="467" alt="Screenshot 2026-04-14 at 15 35 28" src="https://github.com/user-attachments/assets/1cc6453a-74ad-45be-b855-ecccc28fc20f" /> (cherry picked from commit a34ca6b)
💔 Some backports could not be created
Note: Successful backport PRs will be merged automatically after passing CI. Manual backportTo create the backport manually run: Questions ?Please refer to the Backport tool documentation |
# Backport This will backport the following commits from `main` to `9.3`: - [[scout] move internal api tests to scout steps (#263021)](#263021) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Dzmitry Lemechko","email":"dzmitry.lemechko@elastic.co"},"sourceCommit":{"committedDate":"2026-04-14T23:06:13Z","message":"[scout] move internal api tests to scout steps (#263021)\n\n## Summary\n\nCloses https://github.com/elastic/appex-qa-team/issues/850\n\nWith selective testing enabled, it makes sense to move scout internal\napi tests running together with other tests, so that they can be skipped\ncompletely when there are 0 affected modules.\n\nI also noticed that running tests without building plugins might be not\nreliable, so let's run internal tests against build as any other tests.\n\n<img width=\"1132\" height=\"467\" alt=\"Screenshot 2026-04-14 at 15 35 28\"\nsrc=\"https://github.com/user-attachments/assets/1cc6453a-74ad-45be-b855-ecccc28fc20f\"\n/>","sha":"a34ca6bba5d837d4c1b5b230d830f9669897344f","branchLabelMapping":{"^v9.5.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","backport:all-open","v9.5.0"],"title":"[scout] move internal api tests to scout steps","number":263021,"url":"https://github.com/elastic/kibana/pull/263021","mergeCommit":{"message":"[scout] move internal api tests to scout steps (#263021)\n\n## Summary\n\nCloses https://github.com/elastic/appex-qa-team/issues/850\n\nWith selective testing enabled, it makes sense to move scout internal\napi tests running together with other tests, so that they can be skipped\ncompletely when there are 0 affected modules.\n\nI also noticed that running tests without building plugins might be not\nreliable, so let's run internal tests against build as any other tests.\n\n<img width=\"1132\" height=\"467\" alt=\"Screenshot 2026-04-14 at 15 35 28\"\nsrc=\"https://github.com/user-attachments/assets/1cc6453a-74ad-45be-b855-ecccc28fc20f\"\n/>","sha":"a34ca6bba5d837d4c1b5b230d830f9669897344f"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.5.0","branchLabelMappingKey":"^v9.5.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/263021","number":263021,"mergeCommit":{"message":"[scout] move internal api tests to scout steps (#263021)\n\n## Summary\n\nCloses https://github.com/elastic/appex-qa-team/issues/850\n\nWith selective testing enabled, it makes sense to move scout internal\napi tests running together with other tests, so that they can be skipped\ncompletely when there are 0 affected modules.\n\nI also noticed that running tests without building plugins might be not\nreliable, so let's run internal tests against build as any other tests.\n\n<img width=\"1132\" height=\"467\" alt=\"Screenshot 2026-04-14 at 15 35 28\"\nsrc=\"https://github.com/user-attachments/assets/1cc6453a-74ad-45be-b855-ecccc28fc20f\"\n/>","sha":"a34ca6bba5d837d4c1b5b230d830f9669897344f"}}]}] BACKPORT--> Co-authored-by: Dzmitry Lemechko <dzmitry.lemechko@elastic.co>
# Backport This will backport the following commits from `main` to `9.4`: - [[scout] move internal api tests to scout steps (#263021)](#263021) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Dzmitry Lemechko","email":"dzmitry.lemechko@elastic.co"},"sourceCommit":{"committedDate":"2026-04-14T23:06:13Z","message":"[scout] move internal api tests to scout steps (#263021)\n\n## Summary\n\nCloses https://github.com/elastic/appex-qa-team/issues/850\n\nWith selective testing enabled, it makes sense to move scout internal\napi tests running together with other tests, so that they can be skipped\ncompletely when there are 0 affected modules.\n\nI also noticed that running tests without building plugins might be not\nreliable, so let's run internal tests against build as any other tests.\n\n<img width=\"1132\" height=\"467\" alt=\"Screenshot 2026-04-14 at 15 35 28\"\nsrc=\"https://github.com/user-attachments/assets/1cc6453a-74ad-45be-b855-ecccc28fc20f\"\n/>","sha":"a34ca6bba5d837d4c1b5b230d830f9669897344f","branchLabelMapping":{"^v9.5.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","backport:all-open","v9.5.0"],"title":"[scout] move internal api tests to scout steps","number":263021,"url":"https://github.com/elastic/kibana/pull/263021","mergeCommit":{"message":"[scout] move internal api tests to scout steps (#263021)\n\n## Summary\n\nCloses https://github.com/elastic/appex-qa-team/issues/850\n\nWith selective testing enabled, it makes sense to move scout internal\napi tests running together with other tests, so that they can be skipped\ncompletely when there are 0 affected modules.\n\nI also noticed that running tests without building plugins might be not\nreliable, so let's run internal tests against build as any other tests.\n\n<img width=\"1132\" height=\"467\" alt=\"Screenshot 2026-04-14 at 15 35 28\"\nsrc=\"https://github.com/user-attachments/assets/1cc6453a-74ad-45be-b855-ecccc28fc20f\"\n/>","sha":"a34ca6bba5d837d4c1b5b230d830f9669897344f"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.5.0","branchLabelMappingKey":"^v9.5.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/263021","number":263021,"mergeCommit":{"message":"[scout] move internal api tests to scout steps (#263021)\n\n## Summary\n\nCloses https://github.com/elastic/appex-qa-team/issues/850\n\nWith selective testing enabled, it makes sense to move scout internal\napi tests running together with other tests, so that they can be skipped\ncompletely when there are 0 affected modules.\n\nI also noticed that running tests without building plugins might be not\nreliable, so let's run internal tests against build as any other tests.\n\n<img width=\"1132\" height=\"467\" alt=\"Screenshot 2026-04-14 at 15 35 28\"\nsrc=\"https://github.com/user-attachments/assets/1cc6453a-74ad-45be-b855-ecccc28fc20f\"\n/>","sha":"a34ca6bba5d837d4c1b5b230d830f9669897344f"}}]}] BACKPORT--> Co-authored-by: Dzmitry Lemechko <dzmitry.lemechko@elastic.co>
## Summary Closes elastic/appex-qa-team#850 With selective testing enabled, it makes sense to move scout internal api tests running together with other tests, so that they can be skipped completely when there are 0 affected modules. I also noticed that running tests without building plugins might be not reliable, so let's run internal tests against build as any other tests. <img width="1132" height="467" alt="Screenshot 2026-04-14 at 15 35 28" src="https://github.com/user-attachments/assets/1cc6453a-74ad-45be-b855-ecccc28fc20f" /> (cherry picked from commit a34ca6b) # Conflicts: # .buildkite/scout_ci_config.yml # .buildkite/scripts/steps/test/scout/test_run_builder.sh
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
) # Backport This will backport the following commits from `main` to `8.19`: - [[scout] move internal api tests to scout steps (#263021)](#263021) <!--- Backport version: 11.0.1 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Dzmitry Lemechko","email":"dzmitry.lemechko@elastic.co"},"sourceCommit":{"committedDate":"2026-04-14T23:06:13Z","message":"[scout] move internal api tests to scout steps (#263021)\n\n## Summary\n\nCloses https://github.com/elastic/appex-qa-team/issues/850\n\nWith selective testing enabled, it makes sense to move scout internal\napi tests running together with other tests, so that they can be skipped\ncompletely when there are 0 affected modules.\n\nI also noticed that running tests without building plugins might be not\nreliable, so let's run internal tests against build as any other tests.\n\n<img width=\"1132\" height=\"467\" alt=\"Screenshot 2026-04-14 at 15 35 28\"\nsrc=\"https://github.com/user-attachments/assets/1cc6453a-74ad-45be-b855-ecccc28fc20f\"\n/>","sha":"a34ca6bba5d837d4c1b5b230d830f9669897344f","branchLabelMapping":{"^v9.5.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","backport:all-open","v9.4.0","v9.5.0","v9.3.4"],"title":"[scout] move internal api tests to scout steps","number":263021,"url":"https://github.com/elastic/kibana/pull/263021","mergeCommit":{"message":"[scout] move internal api tests to scout steps (#263021)\n\n## Summary\n\nCloses https://github.com/elastic/appex-qa-team/issues/850\n\nWith selective testing enabled, it makes sense to move scout internal\napi tests running together with other tests, so that they can be skipped\ncompletely when there are 0 affected modules.\n\nI also noticed that running tests without building plugins might be not\nreliable, so let's run internal tests against build as any other tests.\n\n<img width=\"1132\" height=\"467\" alt=\"Screenshot 2026-04-14 at 15 35 28\"\nsrc=\"https://github.com/user-attachments/assets/1cc6453a-74ad-45be-b855-ecccc28fc20f\"\n/>","sha":"a34ca6bba5d837d4c1b5b230d830f9669897344f"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"9.4","label":"v9.4.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/263219","number":263219,"state":"MERGED","mergeCommit":{"sha":"6934b187917412faff8668be7e490602d4b20d17","message":"[9.4] [scout] move internal api tests to scout steps (#263021) (#263219)\n\n# Backport\n\nThis will backport the following commits from `main` to `9.4`:\n- [[scout] move internal api tests to scout steps\n(#263021)](https://github.com/elastic/kibana/pull/263021)\n\n\n\n### Questions ?\nPlease refer to the [Backport tool\ndocumentation](https://github.com/sorenlouv/backport)\n\n\n\nCo-authored-by: Dzmitry Lemechko <dzmitry.lemechko@elastic.co>"}},{"branch":"main","label":"v9.5.0","branchLabelMappingKey":"^v9.5.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/263021","number":263021,"mergeCommit":{"message":"[scout] move internal api tests to scout steps (#263021)\n\n## Summary\n\nCloses https://github.com/elastic/appex-qa-team/issues/850\n\nWith selective testing enabled, it makes sense to move scout internal\napi tests running together with other tests, so that they can be skipped\ncompletely when there are 0 affected modules.\n\nI also noticed that running tests without building plugins might be not\nreliable, so let's run internal tests against build as any other tests.\n\n<img width=\"1132\" height=\"467\" alt=\"Screenshot 2026-04-14 at 15 35 28\"\nsrc=\"https://github.com/user-attachments/assets/1cc6453a-74ad-45be-b855-ecccc28fc20f\"\n/>","sha":"a34ca6bba5d837d4c1b5b230d830f9669897344f"}},{"branch":"9.3","label":"v9.3.4","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/263218","number":263218,"state":"MERGED","mergeCommit":{"sha":"17444c9abc507f13d4ee90a83411844eec9b0837","message":"[9.3] [scout] move internal api tests to scout steps (#263021) (#263218)\n\n# Backport\n\nThis will backport the following commits from `main` to `9.3`:\n- [[scout] move internal api tests to scout steps\n(#263021)](https://github.com/elastic/kibana/pull/263021)\n\n\n\n### Questions ?\nPlease refer to the [Backport tool\ndocumentation](https://github.com/sorenlouv/backport)\n\n\n\nCo-authored-by: Dzmitry Lemechko <dzmitry.lemechko@elastic.co>"}}]}] BACKPORT-->
## Summary Closes elastic/appex-qa-team#850 With selective testing enabled, it makes sense to move scout internal api tests running together with other tests, so that they can be skipped completely when there are 0 affected modules. I also noticed that running tests without building plugins might be not reliable, so let's run internal tests against build as any other tests. <img width="1132" height="467" alt="Screenshot 2026-04-14 at 15 35 28" src="https://github.com/user-attachments/assets/1cc6453a-74ad-45be-b855-ecccc28fc20f" /> (cherry picked from commit a34ca6b) # Conflicts: # .buildkite/scripts/steps/test/scout/test_run_builder.sh
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
# Backport This will backport the following commits from `main` to `9.2`: - [[scout] move internal api tests to scout steps (#263021)](#263021) <!--- Backport version: 11.0.1 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Dzmitry Lemechko","email":"dzmitry.lemechko@elastic.co"},"sourceCommit":{"committedDate":"2026-04-14T23:06:13Z","message":"[scout] move internal api tests to scout steps (#263021)\n\n## Summary\n\nCloses https://github.com/elastic/appex-qa-team/issues/850\n\nWith selective testing enabled, it makes sense to move scout internal\napi tests running together with other tests, so that they can be skipped\ncompletely when there are 0 affected modules.\n\nI also noticed that running tests without building plugins might be not\nreliable, so let's run internal tests against build as any other tests.\n\n<img width=\"1132\" height=\"467\" alt=\"Screenshot 2026-04-14 at 15 35 28\"\nsrc=\"https://github.com/user-attachments/assets/1cc6453a-74ad-45be-b855-ecccc28fc20f\"\n/>","sha":"a34ca6bba5d837d4c1b5b230d830f9669897344f","branchLabelMapping":{"^v9.5.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","backport:all-open","v9.4.0","v9.5.0","v9.3.4","v8.19.15"],"title":"[scout] move internal api tests to scout steps","number":263021,"url":"https://github.com/elastic/kibana/pull/263021","mergeCommit":{"message":"[scout] move internal api tests to scout steps (#263021)\n\n## Summary\n\nCloses https://github.com/elastic/appex-qa-team/issues/850\n\nWith selective testing enabled, it makes sense to move scout internal\napi tests running together with other tests, so that they can be skipped\ncompletely when there are 0 affected modules.\n\nI also noticed that running tests without building plugins might be not\nreliable, so let's run internal tests against build as any other tests.\n\n<img width=\"1132\" height=\"467\" alt=\"Screenshot 2026-04-14 at 15 35 28\"\nsrc=\"https://github.com/user-attachments/assets/1cc6453a-74ad-45be-b855-ecccc28fc20f\"\n/>","sha":"a34ca6bba5d837d4c1b5b230d830f9669897344f"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"9.4","label":"v9.4.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/263219","number":263219,"state":"MERGED","mergeCommit":{"sha":"6934b187917412faff8668be7e490602d4b20d17","message":"[9.4] [scout] move internal api tests to scout steps (#263021) (#263219)\n\n# Backport\n\nThis will backport the following commits from `main` to `9.4`:\n- [[scout] move internal api tests to scout steps\n(#263021)](https://github.com/elastic/kibana/pull/263021)\n\n\n\n### Questions ?\nPlease refer to the [Backport tool\ndocumentation](https://github.com/sorenlouv/backport)\n\n\n\nCo-authored-by: Dzmitry Lemechko <dzmitry.lemechko@elastic.co>"}},{"branch":"main","label":"v9.5.0","branchLabelMappingKey":"^v9.5.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/263021","number":263021,"mergeCommit":{"message":"[scout] move internal api tests to scout steps (#263021)\n\n## Summary\n\nCloses https://github.com/elastic/appex-qa-team/issues/850\n\nWith selective testing enabled, it makes sense to move scout internal\napi tests running together with other tests, so that they can be skipped\ncompletely when there are 0 affected modules.\n\nI also noticed that running tests without building plugins might be not\nreliable, so let's run internal tests against build as any other tests.\n\n<img width=\"1132\" height=\"467\" alt=\"Screenshot 2026-04-14 at 15 35 28\"\nsrc=\"https://github.com/user-attachments/assets/1cc6453a-74ad-45be-b855-ecccc28fc20f\"\n/>","sha":"a34ca6bba5d837d4c1b5b230d830f9669897344f"}},{"branch":"9.3","label":"v9.3.4","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/263218","number":263218,"state":"MERGED","mergeCommit":{"sha":"17444c9abc507f13d4ee90a83411844eec9b0837","message":"[9.3] [scout] move internal api tests to scout steps (#263021) (#263218)\n\n# Backport\n\nThis will backport the following commits from `main` to `9.3`:\n- [[scout] move internal api tests to scout steps\n(#263021)](https://github.com/elastic/kibana/pull/263021)\n\n\n\n### Questions ?\nPlease refer to the [Backport tool\ndocumentation](https://github.com/sorenlouv/backport)\n\n\n\nCo-authored-by: Dzmitry Lemechko <dzmitry.lemechko@elastic.co>"}},{"branch":"8.19","label":"v8.19.15","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/263297","number":263297,"state":"MERGED","mergeCommit":{"sha":"91890b4c461cf00a2116db60626a3fd5a541aeb8","message":"[8.19] [scout] move internal api tests to scout steps (#263021) (#263297)\n\n# Backport\n\nThis will backport the following commits from `main` to `8.19`:\n- [[scout] move internal api tests to scout steps\n(#263021)](https://github.com/elastic/kibana/pull/263021)\n\n\n\n### Questions ?\nPlease refer to the [Backport tool\ndocumentation](https://github.com/sorenlouv/backport)\n\n"}}]}] BACKPORT-->
Summary
Closes https://github.com/elastic/appex-qa-team/issues/850
With selective testing enabled, it makes sense to move scout internal api tests running together with other tests, so that they can be skipped completely when there are 0 affected modules.
I also noticed that running tests without building plugins might be not reliable, so let's run internal tests against build as any other tests.